与自然图像相比，医学图像很难获取，标签成本很高。作为一种无监督的学习方法，对比学习可以更有效地利用未标记的医学图像。在本文中，我们使用了一种基于变压器的对比学习方法，并通过转移学习创新了对比度学习网络。然后，将输出模型转移到下游腮腺分割任务，该任务改善了测试集上腮腺分割模型的性能。改善的DSC为89.60％，MPA为99.36％，MIOU为85.11％，HD为2.98。与使用监督学习模型作为腮腺分割网络的预训练模型的结果相比，所有四个指标均显示出显着改善。此外，我们发现，通过对比度学习模型对细分网络的改进主要在编码器部分中，因此本文还试图为解码器部分构建对比度学习网络，并讨论了在构建过程中遇到的问题。

预计变形量子算法将展示量子计算在近期嘈杂量子计算机上的优点。然而，由于算法的大小增加，训练这种变分量子算法遭受梯度消失。以前的工作无法处理由现实量子硬件的必然噪声效应引起的渐变消失。在本文中，我们提出了一种新颖的培训方案，以减轻这种噪声引起的渐变消失。我们首先介绍一种新的成本函数，其中通过在截断的子空间中使用无意程可观察来显着增强梯度。然后，我们证明可以通过从新的成本函数与梯度优化原始成本函数来达到相同的最小值。实验表明，我们的新培训方案对于各种任务的主要变分量子算法非常有效。

Spatiotemporal and motion features are two complementary and crucial information for video action recognition. Recent state-of-the-art methods adopt a 3D CNN stream to learn spatiotemporal features and another flow stream to learn motion features. In this work, we aim to efficiently encode these two features in a unified 2D framework. To this end, we first propose an STM block, which contains a Channel-wise SpatioTemporal Module (CSTM) to present the spatiotemporal features and a Channel-wise Motion Module (CMM) to efficiently encode motion features. We then replace original residual blocks in the ResNet architecture with STM blcoks to form a simple yet effective STM network by introducing very limited extra computation cost. Extensive experiments demonstrate that the proposed STM network outperforms the state-of-the-art methods on both temporal-related datasets (i.e., Something-Something v1 & v2 and Jester) and scene-related datasets (i.e., Kinetics-400, UCF-101, and HMDB-51) with the help of encoding spatiotemporal and motion features together. * The work was done during an internship at SenseTime.

As one of the prevalent methods to achieve automation systems, Imitation Learning (IL) presents a promising performance in a wide range of domains. However, despite the considerable improvement in policy performance, the corresponding research on the explainability of IL models is still limited. Inspired by the recent approaches in explainable artificial intelligence methods, we proposed a model-agnostic explaining framework for IL models called R2RISE. R2RISE aims to explain the overall policy performance with respect to the frames in demonstrations. It iteratively retrains the black-box IL model from the randomized masked demonstrations and uses the conventional evaluation outcome environment returns as the coefficient to build an importance map. We also conducted experiments to investigate three major questions concerning frames' importance equality, the effectiveness of the importance map, and connections between importance maps from different IL models. The result shows that R2RISE successfully distinguishes important frames from the demonstrations.

A storyboard is a roadmap for video creation which consists of shot-by-shot images to visualize key plots in a text synopsis. Creating video storyboards however remains challenging which not only requires association between high-level texts and images, but also demands for long-term reasoning to make transitions smooth across shots. In this paper, we propose a new task called Text synopsis to Video Storyboard (TeViS) which aims to retrieve an ordered sequence of images to visualize the text synopsis. We construct a MovieNet-TeViS benchmark based on the public MovieNet dataset. It contains 10K text synopses each paired with keyframes that are manually selected from corresponding movies by considering both relevance and cinematic coherence. We also present an encoder-decoder baseline for the task. The model uses a pretrained vision-and-language model to improve high-level text-image matching. To improve coherence in long-term shots, we further propose to pre-train the decoder on large-scale movie frames without text. Experimental results demonstrate that our proposed model significantly outperforms other models to create text-relevant and coherent storyboards. Nevertheless, there is still a large gap compared to human performance suggesting room for promising future work.

With the rapid deployment of graph neural networks (GNNs) based techniques into a wide range of applications such as link prediction, node classification, and graph classification the explainability of GNNs has become an indispensable component for predictive and trustworthy decision-making. Thus, it is critical to explain why graph neural network (GNN) makes particular predictions for them to be believed in many applications. Some GNNs explainers have been proposed recently. However, they lack to generate accurate and real explanations. To mitigate these limitations, we propose GANExplainer, based on Generative Adversarial Network (GAN) architecture. GANExplainer is composed of a generator to create explanations and a discriminator to assist with the Generator development. We investigate the explanation accuracy of our models by comparing the performance of GANExplainer with other state-of-the-art methods. Our empirical results on synthetic datasets indicate that GANExplainer improves explanation accuracy by up to 35\% compared to its alternatives.

Spatial-temporal (ST) graph modeling, such as traffic speed forecasting and taxi demand prediction, is an important task in deep learning area. However, for the nodes in graph, their ST patterns can vary greatly in difficulties for modeling, owning to the heterogeneous nature of ST data. We argue that unveiling the nodes to the model in a meaningful order, from easy to complex, can provide performance improvements over traditional training procedure. The idea has its root in Curriculum Learning which suggests in the early stage of training models can be sensitive to noise and difficult samples. In this paper, we propose ST-Curriculum Dropout, a novel and easy-to-implement strategy for spatial-temporal graph modeling. Specifically, we evaluate the learning difficulty of each node in high-level feature space and drop those difficult ones out to ensure the model only needs to handle fundamental ST relations at the beginning, before gradually moving to hard ones. Our strategy can be applied to any canonical deep learning architecture without extra trainable parameters, and extensive experiments on a wide range of datasets are conducted to illustrate that, by controlling the difficulty level of ST relations as the training progresses, the model is able to capture better representation of the data and thus yields better generalization.

变压器验证引起了机器学习研究和行业的越来越多的关注。它正式验证了变压器对对抗性攻击的鲁棒性，例如用同义词交换单词。但是，由于以中线为中心的计算，变压器验证的性能仍然不令人满意，这与标准神经网络有显着差异。在本文中，我们提出了信仰，这是用于GPU的变压器验证的有效框架。我们首先提出一个语义意识的计算图转换，以识别语义信息，例如变压器验证中的结合计算。我们利用此类语义信息，以在计算图级别启用有效的内核融合。其次，我们提出了一个验证专门的内核手工艺品，以有效地将变压器验证映射到现代GPU。该手工艺者利用了一组GPU硬件支持，以加速通常是内存密集型的验证专业操作。第三，我们提出了一个专家指导的自动调整，以纳入有关GPU后端的专家知识，以促进大型搜索空间探索。广泛的评估表明，Faith在最先进的框架上实现了$ 2.1 \ times $至$ 3.4 \ times $（$ 2.6 \ times $）的加速。

大型语言模型（LLM）从人类的指示中解开了任务计划的新功能。但是，事先尝试将LLMS应用于现实世界的机器人任务受到周围场景中缺乏接地的限制。在本文中，我们开发了NLMAP，这是一个开放式摄影和可查询场景表示，以解决此问题。 NLMAP是一个框架，可以将上下文信息收集到LLM计划者中，从而在生成上下文条件条件计划之前，可以在场景中查看和查询可用的对象。 NLMAP首先使用视觉语言模型（VLM）建立自然语言可查询场景表示。基于LLM的对象建议模块解析指令并提出涉及的对象，以查询场景表示以获取对象可用性和位置。然后，LLM规划师计划提供有关场景的此类信息。 NLMAP允许机器人在没有固定的对象列表或可执行选项的情况下操作，从而使真实的机器人操作无法通过以前的方法实现。项目网站：https：//nlmap-saycan.github.io

图形神经网络（GNN）的输入图的大小不断增加，突显了使用多GPU平台的需求。但是，由于计算不平衡和效率较低的通信，现有的多GPU GNN解决方案遭受了劣质性能。为此，我们提出了MGG，这是一种新型的系统设计，可以通过以GPU为中心的软件管道在多GPU平台上加速GNN。 MGG探讨了通过细粒度计算通信管道中隐藏GNN工作负载中远程内存访问延迟的潜力。具体而言，MGG引入了管​​道感知工作负载管理策略和混合数据布局设计，以促进通信局限性重叠。 MGG实现以优化的管道为中心的内核。它包括工作负载交织和基于经经的映射，以进行有效的GPU内核操作管道和专门的内存设计以及优化，以更好地数据访问性能。此外，MGG还结合了轻巧的分析建模和优化启发式方法，以动态提高运行时不同设置的GNN执行性能。全面的实验表明，MGG在各种GNN设置上的最先进的多GPU系统要比最先进的多GPU系统：平均比具有统一虚拟内存设计的多GPU系统快3.65倍，平均比DGCL框架快7.38倍。